{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Eager Execution Tutorial: Basics",
      "version": "0.3.2",
      "views": {},
      "default_view": {},
      "provenance": [
        {
          "file_id": "0B0kLcpwLFwKEVm9XNkFueGk4bTg",
          "timestamp": 1504118841551
        }
      ]
    }
  },
  "cells": [
    {
      "metadata": {
        "id": "U9i2Dsh-ziXr",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Eager Execution Tutorial: Basics\n",
        "\n",
        "This notebook introduces the basics of using TensorFlow's eager execution capabilities. It covers concepts such as:\n",
        "\n",
        "* Importing required packages\n",
        "* Enabling eager execution\n",
        "* Creating and using TensorFlow Tensors and Variables\n",
        "* Using TensorFlow interactively\n",
        "* Using GPUs with eager execution enabled\n",
        "\n",
        "This notebook does *not* cover modeling topics, such as gradients."
      ]
    },
    {
      "metadata": {
        "id": "z1JcS5iBXMRO",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 1: Import Eager\n",
        "\n",
        "The key imports for eager execution are the following:"
      ]
    },
    {
      "metadata": {
        "id": "RlIWhyeLoYnG",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "cellView": "code"
      },
      "cell_type": "code",
      "source": [
        "# Import TensorFlow.\n",
        "import tensorflow as tf\n",
        "\n",
        "# Import TensorFlow eager execution support (subject to future changes).\n",
        "tfe = tf.contrib.eager"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "H9UySOPLXdaw",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 2: Enable eager execution\n",
        "\n",
        "All future TensorFlow calls will execute the\n",
        "underlying TensorFlow ops immediately:"
      ]
    },
    {
      "metadata": {
        "id": "WPTUfGq6kJ5w",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "cellView": "code"
      },
      "cell_type": "code",
      "source": [
        "tf.enable_eager_execution()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "twBfWd5xyu_d",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 3: Interactively Use TensorFlow!\n",
        "\n",
        "Now you can call TensorFlow functions and get results, immediately! No more `tf.Sessions`!\n",
        "\n",
        "TensorFlow will automatically wrap native Python types for you with operator overloading for TensorFlow Tensors."
      ]
    },
    {
      "metadata": {
        "id": "ngUe237Wt48W",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "cellView": "code"
      },
      "cell_type": "code",
      "source": [
        "print(tf.add(1, 2))\n",
        "print(tf.add([1, 2], [3, 4]))\n",
        "print(tf.square(5))\n",
        "print(tf.reduce_sum([1, 2, 3]))\n",
        "print(tf.encode_base64(\"hello world\"))\n",
        "print(\"\")\n",
        "\n",
        "x = tf.constant(2)\n",
        "y = tf.constant(3)\n",
        "print(x * y + 1)\n",
        "\n",
        "# Most TensorFlow ops are directly usable with eager execution, giving\n",
        "# results immediately.\n",
        "print(tf.contrib.signal.hamming_window(x * y + 1))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "IDY4WsYRhP81",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Numpy arrays are supported, too:"
      ]
    },
    {
      "metadata": {
        "id": "lCUWzso6mbqR",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "\n",
        "ones = np.ones([3, 3])\n",
        "\n",
        "print(\"numpy 3x3 matrix of 1s:\")\n",
        "print(ones)\n",
        "print(\"\")\n",
        "\n",
        "print(\"Multiplied by 42:\")\n",
        "print(tf.multiply(ones, 42))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "PBNP8yTRfu_X",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 4: Define and Print TensorFlow Variables\n",
        "\n",
        "To define TensorFlow variables, use the `get_variable()` function as follows:"
      ]
    },
    {
      "metadata": {
        "id": "3Twf_Rw-gQFM",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "cellView": "code"
      },
      "cell_type": "code",
      "source": [
        "x = tfe.Variable(0.)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "45G7094TxsMb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Printing TensorFlow Variables"
      ]
    },
    {
      "metadata": {
        "id": "UJBJeZ5XxuwA",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "cellView": "code"
      },
      "cell_type": "code",
      "source": [
        "# This does NOT print the Variable's actual value:\n",
        "print(\"Printing a TensorFlow Variable:\")\n",
        "print(x)\n",
        "print(\"\")\n",
        "\n",
        "\n",
        "print(\"Printing a TensorFlow Variable's value as a numpy array:\")\n",
        "print(x.numpy())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "2njjWHcTpBEn",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Changing a TensorFlow Variable's value\n",
        "\n",
        "To change a TensorFlow Variable's value, use its `.assign()` or `.assign_add()` method:"
      ]
    },
    {
      "metadata": {
        "id": "v3wr6Erbo_hB",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "cell_type": "code",
      "source": [
        "x.assign(42)\n",
        "print(x)\n",
        "\n",
        "x.assign_add(3)\n",
        "print(x)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "uhtynjHVpTB5",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Use a Variable just like any other Tensor"
      ]
    },
    {
      "metadata": {
        "id": "7PbktdnHoehR",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "cell_type": "code",
      "source": [
        "print(x + 3)\n",
        "\n",
        "# This code will broadcast the value across the list of numbers:\n",
        "print(x * [1, 2, 4])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "GVChqwlwy1SI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 5: Debug Errors with Instant Feedback\n",
        "\n",
        "TensorFlow's eager execution helps you identify and debug runtime issues through interactive exploration of code snippets.\n",
        "\n",
        "Below, we'll define a length-4 vector, and attempt two `tf.slice()` operations,\n",
        "one being legal and the other being illegal, leading to a runtime error that is\n",
        "raised immediately."
      ]
    },
    {
      "metadata": {
        "id": "23ap04N0v4k0",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "cellView": "code"
      },
      "cell_type": "code",
      "source": [
        "vector = tf.constant([10.0, 20.0, 30.0, 40.0])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "FCUMsIYxxRRa",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "cellView": "code"
      },
      "cell_type": "code",
      "source": [
        "# Works, because the values of `begin` and `size` (the 2nd and 3rd input\n",
        "# arguments) are within the bound of `vector`.\n",
        "print(tf.slice(vector, [1], [3]))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "T8me2oCNxpFp",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "cellView": "code"
      },
      "cell_type": "code",
      "source": [
        "# The following does NOT work, because the value of `size` (the 3rd\n",
        "# argument) causes the indices to go out of the bounds of `vector`. The\n",
        "# error is raised immediately.\n",
        "try:\n",
        "  print(tf.slice(vector, [1], [4]))\n",
        "except tf.OpError as e:\n",
        "  print(\"Caught error: %s\" % e)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "irxJhAgar84v",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Step 6: Using the GPU\n",
        "\n",
        "You can explicitly place Tensors on the GPU by calling a Tensor's `.gpu()` method. The `.device` property tells you whether the Tensor is backed by CPU or GPU memory.\n",
        "\n",
        "The first operation executing on the GPU may be slow as TensorFlow initializes. Subsequent uses will be much faster."
      ]
    },
    {
      "metadata": {
        "id": "7J4N9baqaKCL",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "cell_type": "code",
      "source": [
        "# Create some Tensors\n",
        "SIZE = 1000\n",
        "tensor = tf.random_normal([SIZE, SIZE])\n",
        "print(tensor.device)\n",
        "\n",
        "\n",
        "if tf.test.is_gpu_available():\n",
        "  gpu_tensor = tensor.gpu()\n",
        "  cpu_tensor = tensor.cpu()\n",
        "else:\n",
        "  print(\"GPU not available.\")\n",
        "  cpu_tensor = tensor"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "4E-2n7VbzY1n",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "cell_type": "code",
      "source": [
        "# Time a CPU-based matrix multiplication\n",
        "\n",
        "print(\"Time to conduct matmul on CPU:\")\n",
        "%time tf.matmul(cpu_tensor, cpu_tensor)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "vbSFW-T5zhZF",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "cell_type": "code",
      "source": [
        "# Time GPU-based matrix multiplications.\n",
        "\n",
        "if tf.test.is_gpu_available():\n",
        "  # First use of the GPU will be slow:\n",
        "  print(\"Time to conduct first matmul on GPU:\")\n",
        "  %time tf.matmul(gpu_tensor, gpu_tensor)\n",
        "  print()\n",
        "\n",
        "  # Subsequent uses are much faster:\n",
        "  print(\"Time to conduct second matmul on GPU:\")\n",
        "  %time tf.matmul(gpu_tensor, gpu_tensor)"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}